TREC 7 Ad Hoc, Speech, and Interactive tracks at MDS/CSIRO

1 Overview For the 1998 round of TREC, the MDS group, long-term participants at the conference, jointly participated with newcomers CSIRO. Together we completed runs in three tracks: ad-hoc, interactive, and speech. 2 Ad-hoc task In TREC-5 we used document retrieval based on arbitrary passages 8, 9], or xed-length passages that could start at any word position. Although far from the best runs in TREC-5, these results were promising, in particular for long documents. In TREC-6 we continued with arbitrary passages, but our main emphasis was on comprehensive factor analysis of successful automatic query expansion and reenements methods in the context of the vector space model 5]. This year we have reened the MG retrieval system to include Rocchio-based relevance feedback. Also, phrase matching has been added. We have continued to use arbitrary passages and combination of evidence for document retrieval. An in-house version of the MG retrieval system has been used for all experiments. All experiments were carried out on an Intel Pentium II (300 Mhz) with a single processor and 256 Mb of physical memory. Queries and documents were matched using the Okapi formulation 13]: sim(q; d) = X t2q^d w d;t wq;t (1) with w d;t : (k1 + 1) f d;t k1 (1 ? b) + b W d avr W d ] + f d;t and wq;t: (k3 + 1) fq;t k3 + fq;t log N ? ft + 0:5 where k1, k3, and b are constants set to 1:2, 1000, and 0:75 respectively , as recommended by the City University group 13]. The value W d is the length of document d in bytes and avr W d is the average document length in the entire collection. The value N is the total number of documents in the collection, ft is the number of documents in which term t occurs, and fx;t is the frequency of term t in either document d or query q. Okapi is not easily adaptable to arbitrary passage ranking because parameters kx and b are tuned to document ranking. Queries and passages are matched using a non-normalised version of the cosine similarity function: sim(q; p) = X t2q^p (wq;t wp;t) (2) with weights that have been shown to be robust and give good retrieval performance 1]: wq;t = (log(fq;t) + 1) log(N ft + 1) and wp;t = log(fp;t) + 1: Automatic relevance feedback was based on the Rocchio …